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- Th MAILING DATE of this communication appears on the cover sheet with the correspondenc address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)S Responsive to communication(s) filed on 09 December 2003 . 
2a)D This action is FINAL. 2b)(3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) [3 Claim(s) 1-5 and 7-20 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) ^ Claim(s) 7-10 is/are allowed. 

6) [3 Claim(s) 1-4 and 14-20 is/are rejected. 

7) [X] Claim(s) 5 and 11-13 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. §§ 119 and 120 

1 2) S Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 

a)KlAII b)D Some*c)D None of: 

1 Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

1 3) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 1 9(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) D The translation of the foreign language provisional application has been received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet (^^^ : ^^^^ / "* 
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DETAILED ACTION 



Continued Examination Under 3 7 CFR 1. 114 
1 . A request for continued examination under 37 CFR 1. 1 14, including the fee'set forth in 37 
CFR 1 . 17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1. 1 14, and the fee set forth in 37 CFR 
1. 17(e) has been timely paid, the finality of the previous Office action has been withdrawn 
pursuant to 37 CFR 1.114. Applicant's Request for Continued Examination (RCE) 
submission filed on 09-December-2003 has been entered. In addition, the "After Final" 
amendment filed on 08-October-2003 has been entered as a preliminary amendment for this 
continued examination. 



Remarks 

2. In response to communications filed on 08-October-2003, claim 6 is cancelled and claims 1, 
5, 7-15, 17, and 19 have been amended per applicant's request. Therefore, claims 1-5 and 7- 
20 are pending in the application. 



Claim Rejections - 35 USC §103 
3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1-3 are rejected under 35 U.S.C. 103(a) as being unpatentable over Ma et al (U.S. 
patent No. 6,347,3 13) in view of Liddv et al (U.S. patent No. 6,304,864), and further in view 
ofHerz (U.S. Patent No. 4,460,036.) 

As to claim 1, Ma et al teaches a multimedia data structure used for an image search, 
comprising: 

(a) information describing at least one feature of a certain image (see column 3, lines 18-23); 

and 

(b) recent user feedback information (see column 3, lines 37-56) based on user relevance 
feedback (see column 6, lines 45-50.) 

Ma et al does not teach: 

(c) whole feedback information based on the user relevance feedback. 

Liddy et al teaches a system for retrieving multimedia information (see Abstract), in which 
she teaches whole feedback information based on the user relevance feedback (see column 12, 
lines 9-24, and see column 13, lines 26-38, where "whole feedback" is read on "relevance 
feedback" on a "periodic time interval".) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the time 
the invention was made to have modified Ma et al to include whole feedback information based 
on the user relevance feedback. 
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It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Ma et al by the teaching of Liddy et al because having 
whole feedback information based on the user relevance feedback, would enable the system to 
define periodic intervals automatically, or as set by the user, to allow accumulation of all 
relevance feedback on a particular object (image) to be captured, in order to categorize the 
objects (image) based on whole user relevance feedback. 

Maetal as modified, still does not teach feedback obtained since formation of the data 
structure. 

Herz teaches a method and system for customized electronic identification of desirable 
objects (see Abstract), in which he teaches feedback obtained since formation of the data 
structure (see column 17, lines 30-40, and see column 19, line 45 through column 20, line 10, 
where "since formation of the data structure" is read on feedback on profile Y during years 1990 
through 1995, as of creation date of profile Y being 1990.) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the time 
the invention was made to have modified Ma et al as modified, to include feedback obtained 
since formation of the data structure. 

It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Ma et al as modified, by the teaching of Herz , because 
including feedback obtained since formation of the data structure, would be able to provide an 
accumulated/historical user feedback on images and electronic objects of interest to users, since 
the creation of such objects, as opposed to only feedback just recently received from users. 
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As to claim 2, Ma et al as modified teaches wherein the recent user feedback information is 
determined for a predetermined time period (see Liddv et al, column 12, lines 13-14, where 
"predetermined time period" is read on "interval may be 15 minutes") or by a predetermined 
frequency (see Liddv et al column 12, lines 21-22, where "predetermined frequency" is read on 
"X number of documents".) 

As to claim 3, Ma et al as modified teaches wherein the recent user feedback information is a 
weight value learned by the user relevance feedback or a similar image information (see Ma et 
al, column 6, lines 49-50, and see column 8, lines 4-10), and the whole feedback information is 
represented by a weight value learned by previous feedback (see Liddv et al . column 13, lines 
30-33.) 

5. Claims 4 and 14-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over Ma et al 
(U.S. patent No. 6,347,3 13) in view of Liddv et al (U.S. patent No. 6,304,864) and further in 
view of Herz (U.S. Patent No. 4,460,036), as applied to claims 1-3 above, and still further in 
view of Cohen (U.S. Patent No. 6,067,539.) 

As to claim 4, Ma et al as modified does not teach the data structure further comprising 
recent user feedback reliability information representing how reliable the recent user 
feedback information is, and whole feedback reliability information representing how 
reliable the whole feedback information is. 

Cohen teaches an intelligent information retrieval system (see Abstract), in which he 
teaches: recent user feedback reliability information representing how reliable the recent 
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user feedback information is (see column 7, lines 51-56), and whole feedback reliability 
information representing how reliable the whole feedback information is (see column 2, lines 
45-64, where "whole feedback" is read on "updating the score with scores received on 
previous message".) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the 
time the invention was made to have modified Ma et al as modified to include recent user 
feedback reliability information representing how reliable the recent user feedback 
information is, and whole feedback reliability information representing how reliable the 
whole feedback information is. 

It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Ma et al as modified, by the teaching of Cohen , 
because recent user feedback reliability information representing how reliable the recent user 
feedback information is, and whole feedback reliability information representing how 
reliable the whole feedback information is, would increase the efficiency and accuracy of the 
entered feedback and allow data (images) with the most relevant/reliable user feedback to 
receive a higher rank/weight for presentation to the user than data (images) with a lower 
reliability score. 



As to claim 14, Ma et al teaches a method of determining weights of image features used for 
an image search based on user relevance feedback (see Abstract), comprising: 
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(a) providing a multimedia data structure (see column 1, lines 5-10) including information 
describing the features of a certain image (see column 3, lines 18-23), and recent user feedback 
information for the image (see column 3, lines 37-56* and see column 6, lines 45-50); 

(b) updating the recent user feedback information (see column 2, lines 28-33); 

(c) determining weights of the image features in proportion to the reliabilities of the recent 
feedback information (see column 6, lines 1 1-18.) 

Ma et al does not teach: whole feedback information for the image; and determining weights 
of whole feedback information, or both the recent feedback information and the whole feedback 
information. 

Liddv et al teaches a system for retrieving multimedia information (see Abstract), in which 
she teaches: whole feedback information for the image (see column 12, lines 9-24, and see 
column 13, lines 26-38, where "whole feedback" is read on "relevance feedback" on a "periodic 
time interval"); and determining weights of whole feedback information, or both the recent 
feedback information and the whole feedback information (see column 13, lines 15-18.) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the time 
the invention was made to have modified Ma et al to include whole feedback information for the 
image; and determining weights of whole feedback information, or both the recent feedback 
information and the whole feedback information. 

It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Ma et al by the teachings of Liddy et al because obtaining 
whole feedback information for the image; and determining weights of whole feedback 
information, or both the recent feedback information and the whole feedback information, would 
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enable the system to define periodic intervals automatically, or as set by the user, to allow 
accumulation of all relevance feedback on a particular object (image) to be captured, in order to 
categorize the objects (image) based on whole user relevance feedback and based on the weights 
(ranking) of relevance of the entered feedback. 

Ma et al as modified, still does not teach feedback based on relevance feedback obtained 
since formation of the data structure. 

Herz teaches a method and system for customized electronic identification of desirable 
objects (see Abstract), in which he teaches feedback obtained since formation of the data 
structure (applicant is kindly directed to remarks and discussions made in claim 1 above.) 

Ma et al as modified still does not teach reliability information corresponding to the recent 
user feedback information and whole feedback information; and updating whole feedback 
information and their reliabilities by learning them in response to the user relevance feedback. 

Cohen teaches an intelligent information retrieval system (see Abstract), in which he teaches 
reliability information corresponding to the recent user feedback information (see column 7, 
lines 51-56), and whole feedback information (see column 2, lines 45-64, where "whole 
feedback" is read on "updating the score with scores received on previous message"); and 
updating whole feedback information and their reliabilities by learning them in response to the 
user relevance feedback (see column 7, lines 36-38.) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the time 
the invention was made to have modified Ma et al as modified to include reliability information 
corresponding to the recent user feedback information and whole feedback information; and 
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updating whole feedback information and their reliabilities by learning them in response to the 
user relevance feedback. 

It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Ma et al as modified, by the teaching of Cohen , because 
reliability information corresponding to the recent user feedback information and whole 
feedback information; and updating whole feedback information and their reliabilities by 
learning them in response to the user relevance feedback, would increase the efficiency and 
accuracy of the entered feedback and allow data (images) with the most relevant/reliable user 
feedback to receive a higher rank/weight for presentation to the user than data (images) with a 
lower reliability score. 

As to claim 15, Ma et al as modified teaches wherein the recent user feedback information is 
represented by a weight value learned by the user relevance feedback or a similar image 
information (see Ma et al . column 6, lines 49-50, and see column 8, lines 4-10), and the whole 
feedback information is represented by a weight value learned by feedback (see Liddy et al , 
column 13, lines 30-33) given since formation of the data structure (see Herz, column 17, lines 
30-40, and see column 19, line 45 through column 20, line 10, where "since formation of the 
data structure" is read on feedback on profile Y during years 1990 through 1995, as of creation 
date of profile Y being 1990.) 



As to claim 16, Ma et al as modified teaches wherein the reliability of the recent user 
feedback information (see Cohen , column 7, lines 51-56) is determined in proportion to a 
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consistency of a recently used pattern or feedback (see Liddy et al column 3, line 64 trough 
column 4, line 9.) 

As to claim 17, Ma et al as modified teaches wherein the reliability of the whole feedback 
information is determined in proportion to a number of feedback responses concerned in learning 
(see Cohen , column 9, lines 32-51.) 

As to claim 18, Ma et al as modified teaches wherein the reliability of the whole feedback 
information is responsive to recorded usage records wherein the recorded user usage records 
provide feedback to the reliability of the whole feedback information without user interaction 
(see Cohen , column 4, lines 31-39.) 

As to claim 19, the applicant is directed to the remarks and discussions made in claims 1 and 
14 above. 

As to claim 20, Ma et al as modified teaches wherein the reliability information indicates 
reliability of both the user feedback information (see Cohen , column 7, lines 51-56) and the 
whole feedback information (see Cohen , column 2, lines 45-64, where "whole feedback" is read 
on "updating the score with scores received on previous message.) 
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Allowable Subject Matter 

6. Claims 7-10 are allowed over the prior art made of record. 

7. The following is a statement of reasons for the indication of allowable subject matter: 

The prior art of record, Maetal (U.S. Patent No. 6,347,3 13), Liddv et al (U.S. Patent No. 
6,304,864), Herz (U.S. Patent No. 6,460,036), and Cohen (U.S. Patent No. 6,067,539) do not 
disclose, teach, or suggest the claimed limitations of (in combination with all other features in 
the claims): 

(b) if it is checked that the similar image does not exist in the current queue, inputting the 
similar image to an uppermost space of a queue entrance, setting a number of feedback responses 
of the similar image to "1", and shifting images existing in the queue to lower positions by one 
space; 

(c) if it is checked that the similar image exists in the current queue, increasing the number 
of feedback responses of the similar image, and shifting other images in the queue to upper 
positions by "N"; and 

(d) if any image is shifted to a lowsr position over a size of the queue at the respective steps, 
deleting the image shifted to the lower position from the queue, as claimed in claim 7. 

Claims 8-10 are allowed over the prior art made of record because they are dependents from 
the allowed independent claim 7. 
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8. Claims 5 and 1 1-13 are objected to as being dependent upon a rejected base claim, but would 
be allowable if rewritten in independent form including all of the limitations of the base 
claim and any intervening claims. 

9. The following is a statement of reasons for the indication of allowable subject matter: 

The prior art of record, Ma et al (U.S. Patent No. 6,347,3 13), Liddv et al (U.S. Patent No. 
6,304,864), Herz (U.S. Patent No. 6,460,036), and Cohen (U.S. Patent No. 6,067,539) do not 
disclose, teach, or suggest the claimed limitations of (in combination with all other features in 
the claims): 

wherein the recent user feedback reliability information is expressed by 



where, N is a number of user feedback responses, m is a number of images in a similar image 
list, and n* is a number of user feedback responses given to the i-th image, as claimed in claim 5. 

The prior art of record, Ma et al (U.S. Patent No. 6,347,313), Liddv et al (U.S. Patent No. 
6,304,864), Herz (U.S. Patent No. 6,460,036), and Cohen (U.S. Patent No. 6,067,539) do not 
disclose, teach, or suggest the claimed limitations of (in combination with all other features in 
the claims): 



i - 
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wherein the recent user feedback information is represented as a similar image list, and the 
similar image list has an image list structure which includes a similar image identification, a 
score reflecting the current feedback, and a waiting duration representing a time period between 
a final feedback time point and a present time point, as claimed in claim 1 1 . 

Claims 12-13 are objected to as being dependent upon the objected to dependent claim 1 1 . 

Response to Arguments 
10. Applicant's arguments filed on 08-October-2003 with respect to the rejected claims in view 
of the cited references have been fully considered but they are moot in view of the new 
grounds of rejection. 



11. Any inquiries concerning this communication or earlier communications from the examiner 
should be directed to Tony Mahmoudi whose telephone number is (703) 305-4887. The 
examiner can normally be reached on Mondays-Fridays from 08:00 am to 04:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Dov Popovici, can be reached at (703) 305-3830. 



Conclusion 



January 6, 2004 
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